CLAIMS 



1 . A method for generating a desired view of a real scene from a selected desired 
viewpoint, said method comprising: 

obtaining at least one real scene image from one or more cameras, said one or 
more cameras each having a respective real viewpoint; 

identifying selected objects in said at least one real scene image; 
determining estimates of the positions of the selected objects; 
selecting a desired viewpoint; 

based on the relationship of the selected desired viewpoint to the or each real 
viewpoint, and the estimates of the positions of the selected objects, determining 
positions of the selected objects in said desired view of the scene and rendering a 
view of the scene from the selected desired viewpoint wherein at least some selected 
objects are rendered using image data from at least one real scene source image. 

2. A method according to Claim 1, wherein at least a portion of said rendered 
view is generated without transformation of real images. 

3. A method according to Claim 1, wherein at least a portion of said rendered 
view is generated using image data from a real scene image which is not 
contemporaneous with the image data from which said at least some selected objects 
are rendered 

4. A method according to Claim 1, wherein selected objects are rendered in the 
desired view as projections of real images of those objects obtained from at least one 
real scene image. 

5. A method according to Claim 2, wherein said real images of selected objects 
are transformed, optionally rotated. 

6. A method according to Claim 2, wherein selected objects are rendered in the 
desired view as projections of real images of those objects oriented perpendicular to 
the real camera optical axis. 
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7. A method according to Claim 2, wherein selected objects are rendered in the 
desired view as projections of real images of those objects oriented perpendicular to 
the selected viewpoint optical axis. 

8. A method according to Claim 1, wherein selected objects are rendered in the 
desired view as projections of real images which have been mapped onto 3D surfaces. 

9. A method according to Claim 8, wherein said 3D surfaces are generated in 
response to the outline of the real images of said selected objects obtained from at 
least one real scene image. 

10. A method according to Claim 1, wherein real images of selected objects are 
obtained from said at least one real scene image by a keying process. 

11. A method according to Claim 10, wherein said keying process is a chroma 
keying process or a difference keying process. 

12. A method according to Claim 1, wherein images of selected objects obtained 
from said at least one real scene image are interpolated. 

13. A method according to Claim 1, wherein a set of real scene images are 
obtained from a plurality of cameras having mutually different viewpoints. 

14. A method according to Claim 13, wherein each selected object in the desired 
view is rendered as a projection of a real image of that object extracted from the one 
of said set of real scene images that corresponds to the real viewpoint closest to the 
desired viewpoint. 

15. A method according to Claim 13, wherein each selected object in the desired 
view is rendered using image data from two or more of said set of real scene images. 

16. A method according to Claim 13, wherein projections of real images are 
projections of real images mapped onto 3D surfaces. 

17. A method according to Claim 16, wherein said 3D surfaces are generated from 
the intersections of generalised cones of the outline of a selected object viewed from 
different viewpoints, which generalised cones are the union of visual rays from all 
silhouette points of a selected object. 
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1 8. A method according to Claim 1 3, wherein one or more of said real cameras 
are slave cameras, which are automatically controlled based on camera parameters of 
others of said real cameras. 

19. A method according to Claim 13, wherein said different viewpoints comprises 
at least one elevated viewpoint and at least one low-level viewpoint 

20. A method according to Claim 19, wherein images from said elevated 
viewpoints are used to determine the position of selected objects in a scene and/or 
images from said low-level viewpoints are used to render selected objects in the 
desired view. 

21. A method according to Claim 1, further comprising tracking selected objects 
in one or more sequences of real scene images. 

22. A method according to Claim 21, wherein said object tracking comprises 
obtaining a silhouette of selected objects from a real scene image by keying, and 
analysing changes in shape or position of the silhouette in successive real scene 
images. 

23. A method according to Claim 1, including providing a user interface to allow 
an operator to view one or more real scene images, and to modify an automatic object 
tracking process. 

24. A method according to Claim 23, wherein said user interface additionally 
allows an operator to modify the keying of a selected object in one or more real scene 
images. 

25. Apparatus for generating a desired view of a real scene from a selected desired 
viewpoint, comprising: 

means for obtaining at least one real scene image from one or more cameras, 
the or each camera having a respective real viewpoint; 

means for identifying selected objects in said at least one real scene image; 
means for determining estimates of the positions of the selected objects; 
means for selecting a desired viewpoint; and 
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based on the relationship of the selected desired viewpoint to the or each real 
viewpoint, means for determining positions of the selected objects in said desired 
view of the scene and rendering a view of the scene from the selected desired 
viewpoint wherein at least some selected objects are rendered using image data from 
at least one real scene source image. 

26. A method of monitoring a scene for virtual image generation, said method 
comprising: 

obtaining a set of real scene images from a plurality of cameras having 
mutually different viewpoints; 

using image data from at least a first of said real scene images to derive the 
position of a selected object in the scene; and 

using image data from at least a second of said real scene images to render a 
virtual image of said selected object. 

27. A method according to Claim 26, wherein a first subset of real scene images 
are used to derive position, and a second subset of real scene images are used for 
rendering. 

28. A method according to Claim 26, wherein at least one of said real cameras 
provides an elevated viewpoint, and at least one of said real cameras provides a low- 
level viewpoint, and wherein said first subset of images includes images from at least 
one camera having an elevated viewpoint of the scene, and said second subset 
includes image from at least one camera having a low-level viewpoint of the scene. 

29. A method according to Claim 26, wherein each real camera is located at a 
different lateral orientation around a scene. 

30. A method of controlling a slave camera based on the parameters of at least one 
other camera, said method comprising: 

adjusting the parameters of said slave camera to point and focus at a desired 
point based on the camera parameters of at least one of said other cameras. 

31. A method according to Claim 30, wherein all of the pan, tilt, zoom and focus 
parameters are controlled automatically. 
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32. Apparatus for tracking selected objects in a scene comprising: 

one or more cameras arranged to obtain one or more real scene images; 

image processing means for identifying said selected objects in said one or 
more real scene images; 

means for providing an estimate of the position of said one or more selected 
objects based on their position in the one or more real scene images; 

a user interface adapted to allow an operator to view said estimate of the 
position of selected objects in a real scene image, said user interface including input 
means to allow an operator to modify said estimate. 

33. Apparatus according to Claim 32, wherein real scene images are obtained 
from a plurality of cameras having different view points. 

34. Apparatus according to Claim 33, wherein more than one real scene images 
from different viewpoints are displayed simultaneously, and wherein said estimate is 
indicated graphically on more than one real scene image. 

35. Apparatus according to Claim 32, arranged to allow an operator to select those 
cameras from which real scene images are used to provide said estimate of location. 

36. Apparatus according to Claim 32, arranged to allow an operator to indicate the 
position of one or more selected objects in one or more real scene images. 

37. Apparatus according to Claim 32, arranged to allow an operator to indicate the 
position of one or more selected objects in a first real scene image, and to display an 
estimate of the corresponding position of said one or more objects in at least a second 
real scene image. 

38. Apparatus according to Claim 32, including means for estimating the 
trajectory of a selected object based on an indicated position of the object at a first 
instant, an indicated position of the object at a second instant, the time elapsed 
between said two instants, and physical assumptions of the object's trajectory. 
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